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Increased depth of field for photography. 
Method and means. 

Field of the invention. 

The invention comprises a method to increase the depth of focus (or field) for photography, 
and means for the method. The method constitutes an improvement of the depth of focus at 
use of traditional T / S - technology, "Tilt and Shift", and an increased possibility for 
improved depth of field with other and quite ordinary cameras. The method is mainly aimed 
for digital cameras, where the image is created from a sensor in the camera, instead of on a 
photographic film. By use of present scanning technology however, it is easy to transfer a film 
photo to an electronic image. The method can be used also for those cases. In the following 
the method is mainly presented for the cases, when the picture is created electronically directly 
in the camera. 

In T / S cameras one or both of the lens and the "image plane" are movable. The image plane 
is that plane, where the film or the sensor is positioned. A plane in the scene is sharply 
focused in a "focused plane", which might be localized in front of, back of or cutting the 
image plane. Proportional to the distance between the image plane and the focused plane, the 
blurring grows larger in the image. In T / S cameras the photographer often views the image 
on a focussing screen positioned in the image plane. He controls the camera settings for a 
focused image on the focussing screen. Then he places the film / sensor in this position and 
shoots. 

Photographers use many different ways of working. The profession is often creative, 
sometimes artistic. In practice the photographer puts the camera in position directed to the 
subject, arranges the image plane in the desired perspective, usually vertical, as we are used to 
view the world from a vertical perspective. Tilting an ordinary camera upwards, e g for 
placing a church tower inside the seeker, the camera will produce an image of the tower with a 
tilting perspective, showing a tower tilting or falling in. Parallel vertical lines are no more 
parallel in the image. 

Then the photographer focuses and positions the subject for the image. Here difficulties might 
arise, which the invention can solve. The focal aperture is not only used for control of the light 
strength, but also for increasing the depth of field, e g at product photography. Increased 
exposure can compensate smaller apertures. However long time of exposure implies risks for 
mechanical disturbances blurring the sharpness. 

T / S cameras are used for product photography, and when there is a desire to increase the 
depth of field from what an ordinary camera can offer. A close picture on a product with large 
extension in distance, might be given focus both on the front and back parts, by tilting the 
focal plane, cutting the front as well as the back part of the product image. The tilting might 
be arranged in two ways. One by positioning the film or sensor in the tilting image plane. The 
other by tilting the lens instead, turning the focal plane into the desired angle. Usually a 
vertical image plane is desired, obtaining a vertical perspective, and then there is just the 
method of tilting the lens, until the focal plane is positioned vertically in the desired image 
plane. 

As subjects generally are not extended just in a plane of two dimensions, but are three 
dimensional bodies with varying extensions, also focal aperture reductions are used to 
decrease the angle of refraction from the lens. Thus the image plane can be farther from the 
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real focus, without too large blurring errors. However the subject or the light conditions might 
reduce the possibilities. 

Invention functions. 

The present invention concerns a method to achive sharpness and depth of field for a still 
photo by use of multishots, where different camera settings can be used including aperture 
reduction technique. The method utilizes possibilities for digital image transfer after the shot, 
and the involved processes might be performed using automatic methods to various extents. 
An operator might also in interactions on computer screens, manually improve and control the 
result of the processes. 

The method comprises: 

1. The camera sensor is turned into a suitable focal plane and the photo is shot. 
Then the perspective can be turned into a desired position by image transformation. One way 
to do this is sensing the sensor turning in relation to the desired perspective and feed datas to a 
calculation unit, which calculates how much the image should be turned and makes the 
transformations. E g if the image is tilted an angle u related to the desired vertical plane, then 
the image plane can be turned around its front edge by u. 

Then image points at the distance a from the axis would be scaled to the distance a*cosu from 
the axis. Those points are also moved closer to the lens, a distance a*sinu, which means the 
same reduction of scale as moving the image the corresponding distance closer to the lens, i e 
a distance b to the lens related to (a*sinu + b). This is easily illustrated by drawing radiation 
lines through the optical center to respective image point. Thus turning the image around the 
axis, the points on the image will be projected through simple geometrical rules to the new 
image plane. 

The choice of turning axis can be transfered as turning around another axis plus a transversal 
movementTransversal movements of image elements, changing the distance from the the 
lens, result in a proportional change of scale of the image element. Thus ordinary focusing 
gives rise to size changes. Knowledge about the relative image distance from the lens (the 
optical center) would be used for the size scale. 

When there are defocused image parts, also the defocused area will be enlarged proportional 
to a general image enlargement. A defocused edge would be wider, if special actions is not 
performed. Such actions is included in the invention and will be presented further in the 
section about enlargement. 

Projections of images might be performed as second steps, in turning to the right image 
perspective. The operator might fine tune or control the total process himself, by introducing 
the desired axis and turning. He can also define a certain speed of turn and study the image 
turning until it reaches the desired position, when he can stop the turning or interactively 
change the turn directions, fine tuning the right position. 

The calculations can be performed using a basic image, which has been mapped to an 
increased number of image pixels. Then unnecessary losses of resolution can be avoided 
through the calculations and image processing. 

The control setting of desired focus planes at photography, can be simplified and automatized, 
by the photographer setting the focus successively on those points, through which he wants 
the plane of focus. With known coordinates and focus settings the focus plane can be 
calculated and any turning of lens/sensor also. Simple calculation programs, including the 
camera optics can be stored in the pocket calculator, which can present the result as scale 
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partitions for a turning control. With a PC or other computers the operator can work with the 
image on the computer screen. Programs in the computer can control automatic focus 
measurements and calculation of focal plane and finally control turning and movements of the 
lens (sensor) to the right position. Thus the process can be performed as a production process, 
which is often wanted. 

2. Creation of depth of field by use of several "stills". 

At product photography the scene is often static, i e sveral photos can be shot reproducible. A 
product with relatively large extension in distance, can be photographed from the same 
position with focus set for different parts for the different photos. On the different photos, 
different parts are respective focused and defocused. They are also of different size. 

2.1 Simple addition of image elements. 

A sharp image of the product can be created by adding together focused different image 
elements from respective image. The process is made possible by first changing the image 
scale to the same size, according to 1 above. In many situations the operator might do the 
element selection on a computer screen and determine from which image respective element 
shall be transfered to the final image. 

The process can also be automatic according to the invented methods. The focus for an image 
subarea is compared between images and respective focused image element is selected. The 
division borders between neighbouring image elements are chosen, e g where the defocussing 
is about the same from the different images. Any small deficiences in the division interfaces 
might e g be handled by averaging that area between the respective images. 
Simple product shapes such as a cereal box, might be photographed from a perspective, which 
givesa corner the shortest distance to the camera and other parts successively farther distant. A 
series of photos wih successively changed focus can produce the said focused image elements 
for image element additions. 

The cereal box photographed inclined from a high front position, shows three surfaces; a long, 
a short and an upper side. By tilting the sensor / lens a focused plane might be positioned 
along each of the three appearent surfaces. The three images are transformed by calculations 
to the selected basic image. If the sensor is turned, then the transformation also includes 
perspective returning. 

After the calculations the product image can be combined from the three sharp image 
elements. In practice small deviations might have arisen in positions and sizes, between the 
three different photos. Disturbances might have shaken the positions and controls. So sharp 
borders and edges are identified for the different images and common comers and boundaries 
are matched together, through fine tuned corrections of position and scale for the different 
images. 

The basic image might e g be an image with the right perspective e g a vertical sensor and a 
lens setting giving a focus plane for a product's front, or some other important part of the 
product. This basic "image" can be defined by sensor - lens settings and be the defined "basic 
image" without even being photographed, - also if a photo would have simplified the 
matching of the different photos to the basic image. 

When additional pictures are desired for the integration of a sharp cereal box, then an extra 
focal plane might be positioned along the outmost left and right edges and through the 
diagonal of the upper surface, for helping adjustments of the said surfaces. When also the 
table surface is of interest, on which the box is standing, then a focal plane might be 
positioned also along this surface for an additional photo. 
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The integration of the final image might in this example be relatively simple, as each sharp 
element is easily identified and the defocused areas are substituted by sharp areas from other 
images. 

Recalculations of perspective and scale of size is done to the defined basic image format, with 
the help of known settings for the sensor/lens. Analyses afterwards without knowledge of the 
settings, to find out the the image position relative to another image position, can be 
performed, but it might need some more processing. 

Corrections at the integration of respective sharp image elements are performed using 
common image parts e g edges and corners. Often there are image and text details, which 
simplify the match. If the adjustments of an image wouldn't be good enough, e g dependent 
on a difficult perspective change, then local adjustments can be done for the different image 
elements. The sharp details are positioned in agreements with information from other images, 
and where distances in between is needed to be increased or decreased in a noncoherent way, 
the local area selected for that is preferable an area with slow light and colour variations. 

2.2 Complex products. 

Products might be more or less complicated in structure. There might be arms, poles, holes 
etc. Those can be identified by operators as special areas, "mountains and valleys", where a 
small area might contain large distance variations, and whereby several photos might be 
needed to cover the dynamics. Areas with mountaines and valleys can also be identified 
automatically according to the invention. They are areas, which are characterized in causing 
strong defocussing in consecutive images, although the neighbouring image has sharp 
elements and though the transversal distance is close between the sharp parts of respective 
image. The operator can be alarmed about such and other problem areas by blinking on the 
screen, possibly at the position and with side comments. Often the operator might find focal 
planes, which can be positioned along steep mountains and with additional photos, better 
integrated sharpness can be obtained for such areas. 

If the product has an arm or a pole , then one or more images might be shot with focal planes 
through the arm, such that this subject is shown as a sharp object. In a corresponding way as 
for the cereal box, those images are transformed to a basic image for integration with the other 
images on the product. 

The process for image element identification might start at any photo, but it might be an 
advantage to start with the front or back. In this presentation the start is the front photo, and its 
center element. 

An image area is selected. The area is surveilled considering borders i e border lines and edges 

with fast variations of light strength or colour. The colour resolution might be worse, 

however. The selection method might be the derivative, relative derivative 

(l/p*dp/dq; q=x,y;), difference in light or colour strength, the relative difference of strength, 

etc. Those borders are compared with corresponding values for the photos below, and they are 

defined as belonging to the front photo, when the borders have not better focus in photo 

number 2. The space in between the borders includes surfaces and surfaces limited by borders 

belonging to photo 1 are also preliminarily assigned to this photo until further steps. 

A new adjoining image area is selected and the process is repeated, until the whole image has 

been worked through for the given series of photos, image 1 to n. 



There are alternative search processes preferable in certain cases, e g when there are many 
image elements from different images in the selected search area. Then it might be better 
directly analysing edges of the present photos, and by comparisons identify the respective 
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belongings of the different edges. When the following areas been worked through, the final 
image can be integrated successively. Borders are selected and processed according to 
examples. When a border is an edge between two areas on different focused distances, the 
edge might be assigned to the first area, and the transition position between images might be 
selected at the edge on the second image. 

3. Aspects on focus and sharpness. 

3.1 Sharpness when changing the scale of size, enlargements. 

At enlargements all the image elements are enlarged. Those elements with significant distance 
extensions are enlarged including their defocused parts, which then might be more appearent. 
Defocused borders, e g the border between a red subject and a green background, are imaged 
giving a border width containing various red-green mixtures. For the same defocusing degree, 
the real edge position obtains 50% contributions from each side. A defocused point obtains a 
blurring defocused circle with smeared contributions from the point. Tips, corners and 
irregular shapes are imaged blurred, round smoothed and loosing their detail positions. 
Geometrical objects can be reshaped, by reconstruction of the edges to the corner tips etc. 
When the lack of sharpness is depending of the focusing distances, as in the described 
examples, the focused state is determined by simple optical rules, and according to the 
invention the focused image can be estimated and often also the defocused image borders can 
be corrected. 

Digital photography is based on sensors, usually with smaller dimensions than the final photo 
image. The image processing includes enlargements and then an interest not loosing 
sharpness. Especially at product photography the subject consists of human designed 
structures, i e identifyable geometrical forms. Then there are preconditions to do corrections 
of defocused wellknown detail structures in many cases. 

There is also a natural interest in good-looking product pictures, which emphasizes the 
importance of sharpness. Close up pictures on products enlarge the depth of focus problems, 
and then also the need of tool methods to improve the image quality. ^ 

Depth of field is relatively short at close ups. By reduction of the lens focal distance, the 
subject image decreases approximately proportional, but the relative depth of field might be 
increased. Subsequent magnifications increases the blurring too. Photography at distances, 
using zoom, might give the product a flat impression, which might be negatively experienced. 
With a small sensor and the product object filling the image, there is a need for a lens with a 
small focal distance, giving an angle of view, when looking at the photo, that gives 
associations to the right appearance of the product. Considering that aspect small products 
should be shot at short distances, and then also with short focal distances. 
The invention includes methods, based on knowledge about optical rules, for correction of 
defocussing, and it is based on knowledge about geometries for estimating structures and 
improving the sharpness. 

3.2 Correction of sharpness using optic methods. 

Above, some methods have been presented, by which the image sharpness can be improved eg 
at product photography. Below there is a summary: 

a. Integration of several images on a product, being shot with different camera settings, 
including focus. Perspective turning and scale sizing can be used to match the different 
images to get a sharper image. 
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b. Adding information by use of photo with extra small focal aperture. By that the light will be 
extra reduced and the depth of field increased. Then object details with high light and contrast 
are imaged with larger depth of field. Such images can be used to match different images with 
less depth of field e g in the method in point (a) above. 

c. Adding information from photos being shot with less focal distances. Rechanging the scale 
is simplified when the position is maintained of the subject relative the center of the lens. 
Depth of field might be improved also in those cases. On the other hand the image object grew 
smaller and is not utilizing the resolution at maximum. Also this type of photos can be used in 
the matching of images. 

d. By tilting the lens/sensor, a corresponding tilting focal plane is positioned through the 
product. The image perspective is changed if the sensor is tilted, whereby a subsequent 
perspective turning is performed. This type of images can be used to give information about 
sharpness of elements which might be missing in point (a). 

e. Consecutive photos can be used for optical calculations of the focuse image distance of an 
image element and the real dimensions in this focused position. This method can be used e g 
for image elements that are out of focus on both neighbouring images. 

f. By measuring the defocussing of edges and borders, the focused position of those image 
elements can be calculated and the border blurring be corrected. 

g. Geometries can be inter- and extrapolated from consecutive photos and by that sharpness be 
estimated for defocused distances. The method might be simplified by storing a library of a 
number of usual geometries, and used in comparisons with the image information. Various 
methods, eg best correlation might be used. 



3.3 Some optical relations regarding depth of field. 
Usual optical notations and definitions: 

a, b, f are distances to the lens from the subject , image, focus point. 
H, h are the dimensions (e g the height) of the subject, its image. 

r 0 , tx , are the radius of the focal aperture, the radius of the defocused circle related to a sharp 
point. 

A subject on the distance a + da is focused on the distance b - db. 
Simple optical rules give: 

l/a+l/b=l/f; (1) l/b = (a-f)/a*f ; (2) 

In the same way: 

1 /b(l-db/b)= 1/f- 1 /(a + da) ; (3) 



(2) in (3) gives: 

db / b = da*f / a 2 * 1 / (1 - f/a + da/a) 



(4) 
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and 

db/b~da*f/a 2 ; (5) 
Proportionality gives: 
n = r 0 * db/b ; (6) 
(5) in (6) gives: 
n ~r 0 *da*f/a 2 ; (7) 

For the same subject distance (a) and depth of subject (da) it is obtained that the defocusing is 
decreasing, when the aperture (ro) and the. focal distance (f) are decreasing. When f decreases, 
then the image also decreases, however. The relative unsharpness is: 

rt / h - r 0 * da / (H*a) ; (8) 

n/h*f *da/(H*a*2Q); (9) 

where the aperture Q = f/2r 0 ; 

Thus the reduction of the focal aperture is an important possibility for increase of the depth of 
field. The aperture cannot however be reduced too much, e g the light level arriving at the 
sensor must be lage enough. The density of light for the image, 1(h), at the sensor is 
proportional to the density of light from the subject, 1(H) according to: 

1(h) ~ I(H)*H 2 *r 0 2 / (h*a) 2 ; (10) 

With (10) in (9) it is obtained: 

r-|/h~ (1(h) /1(H)} 05 * da*f/(H*a) ; (11) 

i e if 1(h) cannot be further reduced and the light conditions and level are selected, then a 
reduction of the focal distance presents a possibility to improve the relative focusing. 

3.4. Corrections at integration of the new basic image from the different image photos. 

Photos, shot with the same relative position of the lens optical center relative of the scene 
subject, are simple to transform by perspective turnings and sizing, knowing the settings for 
the lens and sensor of respective photo. By a traditional camera, focal distance settings are the 
only focal plane setting available. That means that the photo series obtaines parallel image 
planes, and no cutting focal planes for the same scene view. Together with the other 
applicable methods, described above, also those cameras can be used for advanced product 
photography, where depth of field and perspective otherwise are problems. 

At multi-photo integration, small deficiences might arise in practice. Tolerances in settings 
and mechanical disturbances and imperfections might cause changes between subsequent 
image shots. Photography with long exposure time might require extra stable basis for camera 
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and subjects, preventing disturbances from blurring the image. Until further actions it is 
presently assumed that each useful" image has good focus within its limited depth of field^ 
Still corrections might be needed, after that the images principally have been transformed to 
the basic image format. Although each image mainly are in the right position, right 
perspective and right size. The following methods can be used for subsequent corrections: 

a Consecutive images with different focus can be corrected, when a sharp element in one 
image corresponds to an unfocused one in neighbouring images. The position of the 
unfocused element can be determined, and by comparisons of detail positions in other images, 
positions, size of scale etc. can be coherently corrected. Common information for images is 
used Calculation processes are faster, when a few well defined elements are selected for the 
comparison process and the elements preferably spread out over the image area. For the right 
image size the matching might be reduced to match the positions of the image elements For 
errors in the scale size, the errors of scale might be obtained by "measurements (calculations) 
of the distances between element positions. At perspective turning the scale of size might be 
different in different parts of the image e g different in the x- and y directions. 

b Common sharp image elements are obtained from cutting focal planes e g for the example 
of the cereal box in point 2.1. By overlapping of common borders and edges the different 
images can be corrected and integrated. 

c Added images can be shot, positioned at cutting focal planes e g to obtain sharp image 
elements in a tilted plane, which then can be recognized as sharp elements in different images. 
Then common references are obtained for and through several images. 

d Images being shot with significantly reduced aperture obtain larger depth of field and can 
be references in corrections of other images, especially for elements with high light and 
contrast strength. 

e. Photos being shot with reduced focal distance can also give larger depth of field in spite of 
subsequent magnifications. 



4. Other examples of the method. 



The sharpness of a product image can also be improved by increasing the resolution of the 
image in added photos, e g a product might be photographed with only parts of it filling the 
image (sensor) area. Then the sensor resolution is better utilized for this part of the total 
pruduct image. The other parts of the product can be photographed with the same or different 
magnifications. Then the different images can be transformed to a basic image, where the 
whole desired image is included, and then with a high resolution. The resolution in the 
subsequent processing of the image is not limited by the condition that the whole image must 
be positioned on the sensor. The computer capacity can be used, creating a computer image 
with much higher resolution than the sensor. The method to integrate the different photos is 
easier, when as earlier, the position of the lens optical center is mainly maintained relative the 
image subject. E g that can be performed by photographing, turning the camera around the 
lens optical center. In the same way as earlier the integration is simplified when the different 
images are overlapped, where well defined elements are contained, and the overlap can also 
be used in comparisons between images for e g corrections. One or more images can also be 
shot with a lens with less focal distance, where essential parts of the whole image are 
contained, and contribute with general information about the basic image. 
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The method of selecting parts of-the subject and photographing those with larger focal 
distances, possibly with the help of zoom, can be used together with the other methods, which 
are used for increased depth of Held. The method might be applied for imaging very small 
details in a product, or for details used in the integration of image areas giving better 
resolution and by that a more accurate match. 



5. Examples on means for performing the improved depth of field. 

In one prefered embodiment, the means consists of units in the camera and units in a 
computation equipment, e g a computer of PC type. The camera units consists of controls, 
which settings can be registered. The settings are for the lens and/or the sensor. For traditional 
cameras with electronic sensor, the distance focusing is made by the lens. T/S cameras might 
have controls for both the lens and the sensor, and those have generally large variability, i e 
they can also do tilting and give focussing and perspectives in many various planes. The 
settings can be controlled and read, and the lens and sensor might also, according to the 
invention, be related to the geometry of the camera design. The values on the optical system as 
focal distance, optical center or corresponding lens descriptions might be valuable in 
subsequent calculations after the photographing. It is simplifying when the position of the 
optical center is known, especially at lens changes between photos. 

In the calculation unit, the computer, there are software which can store several images and 
work with transformations and transfer of images by size of scale, perspective turnings and 
positioning. If the optics and camera setting data are not included together with the image data 
transfer, the operator/photographer might himself read and feed in the values of the settings, 
or the computer makes the evaluation. 

The choice of photo set up is made by the photographer, and he can set the camera manually 
or with help of the computer system. He can e g read the position for the point he will focus 
and read the connected focus setting. By reading three points a focus plane is defined, and a 
computer can help calculating the settings of lens/sensor to establish the desired focal plane. 
The computer can also contain additional program for automating all or various parts of the 
work. The control can also include selection of aperture from light measurements and other 
control of camera functions. 

There are also included programs, which select image areas from respective transfered image, 
possibly with interactions from an operator. 

Programs can be added that, possibly in interaction with an operator, select elements from 
different images, which are the basis for calculations of subsequent corrections of the images, 
including matching the basic image format for integration of focused image elements to the 
new focused basic image. 
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Claims. 



1. A method of photography with electronic image sensors, improving focus of images with 
such extensions that focus of a photo is not obtained over the whole image, but the depth of 
feld is insufficient, then using information from several images or frames, where images are 
shot with different focus, characterized by; shooting the images mainly from the same relative 
position of the lens optical center in relation to the scene subject; and where images (a), being 
shot with different image distances are complemented with images (b), being shot with a 
method for increase of depth of field in relation to said image distances, according to at least 
one of the following methods, photos (bl) with effectively decreased aperture and photos (b2) 
with tilted focal planes relative to (a); and a method for successively integrating image 
information from different images, based on the following steps; where step (si) comprises 
identification of a basic image format according to a choosen image or a defined image 
position, which is related to shot photos; and a step (s2) comprises transfer of the different 
images to the basic image format, by sizing of scale in relation to different focus, and where 
the perspective of an image is turned, this image is projected through geometrical calculations 
to the basic image format; and a step (s3), where a final basic image is built by the help of 
successivly following the distance extension of objects, and with start fr6m a focused image 
element at an image distance, the image area is successively increased (x,y-direction) while 
comparing images for neighbouring distances (z-direction) and when such a second image 
contains focused elements, the final image area is growing by integration, using element areas 
from the second image distance (z) and so on, and a selective order is to start at the front 
image and successively work backwards in subsequent images, and focus comparisons are 
preferably performed directly in the image domain, without any transformations of images to 
the space -frequency domain; and a step (s4) comprises identification of borders, which 
constitutes contours between objects belonging to different focused image distances, and 
which image distance differences are of the order of the depth of field or larger, and image 
element continuity according to (s3) is not contained, and defocused objects (ol) related to a 
first focused image distance, are spread over and disturbing the focused image on the second 
focused image distance, and the disturbance from (ol) is calculated from the sharp position to 
the defocused image distance and subtracted from the focused area of this image; and a step 
(s5) comprises utilizing one-dimensional image elements, as borders, edges and contours 
between objects, geometrical structures, colours etc., obtaining continuity at focus 
comparisons and image distance transfer in the step of (s3); and a step (s6) comprises utilizing 
images (b) to bridge over and to give continuity at image distance transfer, and (bl) especially 
for large depth of field of high light image elements, and by that also being used in (s4) to 
decrease the disturbances of high light defocused elements, and at use of (b2) the image 
integration can be simplified at photography of some objects; and a step (s7) comprises 
utilizing (s6) for fine tuning of step (s2), where (s6) supplies information about relative status 
of focused elements belonging to different image distances (a); and possibly in interaction 
with an operator, who might mark on the images, which parts from respective image he wants 
to have integrated in the final image, which simplifies the processes, then mainly comprising 
geometrical matching and fine tuning of the image elements to the right position and basic 
image format according to (s2). 

2, A method according to claim 1, comprising; at least one tilted focal plane put through the 
subject image with tilting lens or sensor. 



3. A method according to claim 1, comprising; at least one image in a focal plane, which is 
cutting other image planes and then offering elements in different images common sharp 
references. 
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4. A method according to one of the claims 1 - 3, comprising; determining the defocused 
width of borders, edges etc. and subsequently correcting the sharpness also for other image 
elements, selectively by also estimating the image positions of the sharp focused elements. 

5. A method according to one of the claims 1 - 4, comprising; using consecutive images, 
calculating the image plane distance of an image element and its real focused extension at that 
plane, using that for image elements that are defocused in images at both side of the 
theoretically sharp image plane. 

6. A method according to one of the claims 1 - 5, comprising; inter- or extrapolating 
geometrical structures, lines, rectangles, circles etc. from neighbouring image(s) to achive 
focusing in defocused spaces in between, and selectively expanding the method to include 
storing of geometrical structures in a library for comparisons with obtained image elements 
e g by use of correlation, and when good agreements, the geometrical structures are used for 
corrections, the geometric structures possibly also including signs, letters or symbols. 

7. A method according to one of the claims 1 - 6, comprising; correcting at least one of 
position, scale, perspective for different images, performing position comparisons between 
corresponding image elements in at least two images, and selectively using successive 
comparisons between different images, achieving several images related to each other. 

8. A method according to one of the claims 1 - 7, utilizing the sensor resolution better, by 
photographing parts of a product with larger focal distance, the product part covering a larger 
part of the sensor area, comprising; photographing other parts of the product with the same or 
different magnification, where also image distances related to the focal distance might vary, * 
and that different images are transfered to a basic image, where the whole image is covered 
and then with a corresponding higher resolution, where the capacity of the computer is 
utilized, which can give much higher resolution than that of the sensor , covering the whole 
final image, and selectively integrating different images, shot while keeping the position of the 
lens optical center relative to the subject, e g performing that, by turning the camera with the 
optical lens center as the turning point, and simplifying the image matching by overlapping 
images, where well defined elements are included and used at comparisons between images 
for possible corrections, and photographing one or more images with reduced focal distances, 
where essential parts of the total image are included in the same photo, and then might 
contribute with general overview information about the basic image. 

9. Means for performing the tasks according to one of the claims 1 - 8, comprising control 
settings for the cameras, calculation unit for storing of image data with related settings and 
software programs for processing of several images and for calculations of transfer of images 
through change of scales of size, perspectives and positions, comprising; control settings for 
lens and sensor, relating respective position including angles to a geometry in the camera; and 
programs with algorithms, which use settings for respective image photo to transfer the 
images to a selected basic image, which also might be defined by settings, but that must not 
necessarily be a really photographed image; and a program unit, which possibly with the help 
of an operator, selects which image elements to be used from respective transformed image, 
and integrating those to a basic image. 
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10. Means according to claim 10, comprising; a program unit, possibly in interaction with an 
operator, selecting elements from different images, which are the basis for calculations of 
subsequent corrections of the images to the basic image format, achieving better integration of 
the different elements from the different images to the basic image. 



1 1 . Means according to any of the claims 9-10, aiding the photographer / operator with 
controls of the different photo focal plane settings, comprising; means for defining positions 
of points in the image, means for focusing those points and register the focus in the 
calculation unit, means in the calculation unit for calculating subsequent corresponding values 
for lens and / or sensor settings, means for control of the settings for photographing, and 
selectively adding means for storing basic data for respective control in the calculation unit, 
being e g a pocket calculator or a PC, for use at calculations and calibrations of values for 
control of the control means. 

12. Means according to claims 9-11, comprising: means for the photographer to mark points 
on the computer screen image, and computer programs calculating and controlling the camera 
settings automatically by electronic controls. 

13. A method and means according to any of the claims 1-12, where photography film are 
used instead of electronic sensors, comprising; photographing in the corresponding way, and 
subsequently scanning the film photos to obtain electronically stored images, which are 
subsequently processed in the corresponding way. 

14. A method according to any of the claims 1-13, comprising; adding photo with reduced 
focal distance, selectively by zoom or change of lens, where image transformations are 
simplified by keeping the position of the lens optical centers relative the subject, giving a 
smaller image, not fully utilizing the sensor resolution, while depth of field has a potential to 
be increased, although by remagnification of the image also the defocused areas are 
magnified, the defocused elements obtaining improved focus and the image being used for 
adding information and to help matching other images. 
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- AMENDED CLAIMS 

[received by the International Bureau on 15 May 1997 (15 05 97)- 

original claims 1-14 replaced by amended claims 1-16 (5 pages)] 

1. A method to increase the depth of focus, giving improved sharpness of subjects with such 
extensions that sharpness regularly cannot be obtained over the whole picture usin* at least 
two images obtained with different camera settings, which selectively include change of 
focus, change of aperture, in mainly keeping the same position of the lens optical center 
relative to the subject, and combining information from those images to obtain one new 
image, including correction of contours, comprising: 

processing the result of still cameras by computer means, selectively external to the still 
camera functions, using data transfered from the camera on-line or through 
storage media, for principally non-synchronous or off-line processes in the steps 
of; v 

a. matching the different photo-images to the position, scale and perspective of a selected 
basic image in the steps of; 
selecting a basic image; 

identifying and localizing border lines and highly contrasting structures in a number of 
images; 

calculating the geometrical transfer of a number of images to the basic image, selectively 

using position shifts, change of scale and perspective tilting; 
choosing common references for valuating and/or correcting the matching of the images 

using the positions of identified common structures; 
selectively adding at least one image of images, photographed with significantly reduced 

apertures, increasing the depth of focus and images photographed with tilted 

focus planes, cutting at least another image focus plane; 
b. building a new created basic image from the matched images in the steps of; 
using fast variations of light and/or colours as functions of area (x,y) based coordinates for 

determining the best focused images for an analyzed image element; 
comprising for best focused states between images, using border lines and highly 

contrasting structures; 
selecting for the new image, the identified border lines and structures from respective 

focused image; 

selectively watching overlaps and loose ends of the structures; 

selecting transition positions for structures with continous extensions over neighbouring 

images, where the structures are divided belonging to different images- 
selecting border lines for the steps of; ° ' 

analysing a number of border line corridors, narrow pixel areas around borders, 
for differences on each side of the border line between respective images for the 
steps of; ° 

separating between borders in the same focused image and borders between 
different focused images; 

correcting the border line corridor, being a transiton border between different 
images, reducing the spill over from the defocused corridor side to the focused 
corridor side; 

defining transition positions at border lines, where the border line is the border 
between different focused images; 

analysing the areas between border lines for indications of image elements 
with better focus from other images for the step of; 
defining transition positions between structures belonging to different 
neighbouring images; 
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selectively correcting border line corridors and other transition positions 
between images by using information from added images, photographed with 
significantly reduced apertures or with tilted focus planes; 
selecting additional transition positions for the same images through areas containing slow 
variations of light and colour and containing no defined structures from other 

images; . . 

building the new created basic image through the transition positions between different 
images keeping continuity of light level; 

Claim 2. A method according to claim 1 , further comprising: 

analysing the speed of variation in light or colour, selectively using derivatives of, relative 
derivatives of, differences of, and combinations or quotients thereof and of the 
strenght of light or colours. 

Claim 3. A method according to claim 1, further comprising: 

using the methods of reduced aperture or tilted image planes for correction of defocused spill 
over across borders between different focused images in the step of; 
selecting borders, where those methods give a focused image on both sides of 

the border; . , 

selecting this both-side focused image for the border presentation, or one-side 
focused images of the border corridor, after calculating and reducing the 
spill over to respective one-side focused image. 

Claim 4. A method according to claim 1, further comprising: 
building the new basic image successively in the steps of; 

starting at a selected focused image element of a selected image; 

increasing the examined area continously, while comparing with neighbouring 

images* 

making transition of images, when element grows sharper at another image; 
increasing the new basic image area with the area from the present image; 
selectively starting on the front image and successively working towards the 
background. 

Claim 5. A method according to claim 1, further comprising: 
building the new basic image, selecting strategies in the steps of; 

counting density of image border lines; 

selecting low density strategies for large low density areas; 

selecting high density strategies for small high density areas; 
building low density strategies on the steps of; rn- ,.ju n ^. r 

connecting selected border lines of the selected image with affiliated border 

lines by defining transition positions across areas in between; 

transfering the closed image areas to the new basic image; 

Claim 6. A method according to claim 1, further comprising: 

including to the computer means a man machine interface. MMI, for interactions with an 
operator; 
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reducing the complexity of the image processing by an operator selectively performing steps 



defining on the computer screen, which parts from which image to be transfered 

to the new basic image, reducing the processing to mainly image matching and 

final corrections of the parts in position and border defocusing; 

selecting details from different images for the image matching; 

selectively defining strategies for different areas for the new image building; 



of; 
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J- 4TA method according to one of the claims 1 - 3, comprising; determining the defocused 
width of borders, edges etc. and subsequently correcting the sharpness also for other image 
elements, selectively by also estimating the image positions of the sharp focused elements. 

/ 57 A method according to one of the claims I - 4, comprising; using consecutive images, 
calculating the image plane distance of an image element and its real focused extension at that 
plane, using that for image elements that are defocused in images at both side of the 
theoretically sharp image plane. 

-6. A method according to one of the claims 1 - 5, comprising; inter- or extrapolating 
geometrical structures, lines, rectangles, circles etc. from neighbouring image(s) to achive 
focusing in defocused spaces in between, and selectively expanding the method to include 
storing of geometrical structures in a library for comparisons with obtained image elements 
e g by use of correlation, and when good agreements, the geometrical structures are used for 
corrections, the geometric structures possibly also including signs, letters or symbols. 

/ a -77 A method according to one of the claims 1 - 6, comprising; correcting at least one of 
position, scale, perspective for different images, performing position comparisons between 
corresponding image elements in at least two images, and selectively using successive 
comparisons between different images, achieving several images related to each other. 

// & m A method according to one of the claims 1 - 7, utilizing the sensor resolution better, by 
photographing parts of a product with larger focal distance, the product part covering a larger 
part of the sensor area, comprising; photographing other parts of the product with the same or 
different magnification, where also image distances related to the focal distance might vary, 
and that different images are transfered to a basic image, where the whole image is covered 
and then with a corresponding higher resolution, where the capacity of the computer is 
utilized, which can give much higher resolution than that of the sensor , covering the whole 
final image, and selectively integrating different images, shot while keeping the position of the 
lens optical center relative to the subject, e g performing that, by turning the camera with the 
optical lens center as the turning point, and simplifying the image matching by overlapping 
images, where well defined elements are included and used at comparisons between images 
for possible corrections, and photographing one or more images with reduced focal distances, 
where essential parts of the total image are included in the same photo, and then might 
contribute with general overview information about the basic image. 

' ^ ^91 Means for performing the tasks according to one of the claims 1-8, comprising control 
settings for the cameras, calculation unit for storing of image data with related settings and 
software programs for processing of several images and for calculations of transfer of images 
through change of scales of size, perspectives and positions, comprising; control settings for 
lens and sensor, relating respective position including angles to a geometry in the camera; and 
programs with algorithms, which use settings for respective image photo to transfer the 
images to a selected basic image, which also might be defined by settings, but that must not 
necessarily be a really photographed image; and a program unit, which possibly with the help 
of an operator, selects which image elements to be used from respective transformed image, 
and integrating those to a basic image. 
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^rTT Means according to any of the claims 9 - 10, aiding the photographer / operator with 
controls of the different photo focal plane settings, comprising; means for defining positions 
of points in the image, means for focusing those points and register the focus in the 
calculation unit, means in the calculation unit for calculating subsequent corresponding values 
for lens and / or sensor settings, means for control of the settings for photographing, and 
selectively adding means for storing basic data for respective control in the calculation unit, 
being e g a pocket calculator or a PC, for use at calculations and calibrations of values for 
control of the control means. 

*2"Tf4eans according to claims 9 - 1 1, comprising: means for the photographer to mark points 
on the computer screen image, and computer programs calculating and controlling the camera 
settings automatically by electronic controls. 

yS'TA method and means according to any of the claims 1-12, where photography film are 
used instead of electronic sensors, comprising; photographing in the corresponding way, and 
subsequently scanning the film photos to obtain electronically stored images, which are 
subsequently processed in the corresponding way. 

>C"A method according to any of the claims 1-13, comprising; adding photo with reduced 
focal distance, selectively by zoom or change of lens, where image transformations are 
simplified by keeping the position of the lens optical centers relative the subject, giving a 
smaller image, not fully utilizing the sensor resolution, while depth of field has a potential to 
be increased, although by remagnification of the image also the defocused areas are 
magnified, the defocused elements obtaining improved focus and the image being used for 
adding information and to help matching other images. 
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